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ABSTRACT 



This paper describes the development and validation of a 
school -accountability model. The model is based on the assumption that 
schools represent dynamic systems in which a change in one element will 
affect all other elements. The model’s primary elements are educational 
input, processes, and outcomes. The paper examines various accountability 
models and concludes that although these models vary in complexity and 
purpose, their one common element is a reporting system. The model described 
in this report was created to correct the defects in other models. The first 
step in its development was to identify the educational inputs, processes, 
and outcomes of the school. The model provides for the external judgment of a 
school and its programs based on known standards; supplies information to 
school personnel and parents for diagnostic/remedial efforts focused on 
improvement; offers an opportunity for the school to communicate its goals 
and achievement to parents and the public; is culturally fair; and is 
systemic in nature. Future models will be customized for each school, and a 
School Accountability Profile will provide a basis for judging past 
accomplishments and for planning future improvements. The model is currently 
being pilot tested in two of the three largest school districts in Alabama. 
(RJM) 
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Abstract 



The purpose of this paper is to describe the development and validation of a school 
accountability model. A common definition of “accountability” is that someone or some entity 
is held responsible for the performance of an organization. The educational accountability 
movement in this country has been driven primarily by government officials. Often, both the 
methods and criteria used to hold schools accountable are the result of legislation or school 
board actions based on political considerations rather than sound education. It is our belief that 
a school accountability model should have at least five characteristics. It should provide for the 
external judgment of a school and its programs based on known standards (preferably, world 
class), provide information to school personnel and parents for improvement, provide an 
opportunity for the school to communicate its goals and achievements to parents and the public, 
respect diversity, and treat the school as a dynamic system. At best, most accountability systems 
accomplish only the first characteristic. 

The school accountability model proposed in this study is based on the assumption that schools 
represents dynamic systems, a change in one element will impact all other elements. The 
primary elements of the model are educational inputs, processes, and outcomes. The model is 
driven by the school’s goals or desired outcomes. The first task was identifying the indicators 
for these three elements. 

The model is being piloted in two of the three largest school systems in the state. The data will 
be analyzed using structural equation modeling. Structural equation modeling provides for a test 
of the fit of the data to the preconceived model and produces probabilities for associations 
among elements. The model provides input to school administrators and teachers as to impact 
of proposed changes. 
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A SCHOOL ACCOUNTABILITY MODEL 



Accountability is one of the most frequently discussed issues concerning education in the 
popular press. Educational accountability, for the most part, has been taken out of the hands of 
educators and put into the hands of state legislators, lay school boards, and other politicians. 

The reason for this is debatable, but at least one explanation is that educators have defaulted on 
their obligation to provide credible evidence regarding the condition of education to the public. 
The purpose of this paper is to describe the development and validation of a school 
accountability model based on five principles. These are that such models should provide for 
the external judgment of a school and its programs based on known standards (preferably world 
class), provide information to school personnel and parents for improvement, provide an 
opportunity for the school to communicate its goals and achievements to parents and the public, 
respect diversity, and systemic in nature. Most current school accountability systems are based 
on only the first characteristic. The proposed model is designed to address these five principles. 

A common definition of “ accountability” is that someone or some entity is held 
responsible for the performance of an organization. To bring this definition into the context of 
education and the evaluation and assessment paradigm, we explain it this way. “. Measurement ” 
is the process of assigning numbers to individual students to represent their achievement. 

“ Assessment ” is the process of measuring an entire group of students such as a school or school 
system. ” is adding value judgements to these measures. Finally, “ accountability ” places the 
assessment and evaluation information in the context of responsibility for the results. Therefore, 
accountability serves as the umbrella encompassing the other three activities in this hierarchy. 

Background 

The literature on accountability is extensive with much of it being of recent origin. Since 
this paper proposes a new accountability model, the review summarizes the literature concerning 
several accountability models and school reporting systems. This should provide the context in 
which the proposed model was developed. 

Accountability Models 

Two of the most sophisticated accountability models in the literature are based on a 
value-added approach. These are the Tennessee Value Added Assessment System (TVAAS) 
(Baker & Xu, 1995) and the Dallas “value added” system (Webster, Mendro, & Almaguer, 1994; 
Webster & Olsen, 1984). 

The Tennessee Value Added Assessment System. One of the most sophisticated of the 
accountability models is the TVAAS. Originally proposed by McLean and Sanders (1984), 
Sanders has continued to develop and refine the model (e.g., Sanders, 1994; Sanders & Horn, 
1994; Sanders & Stroup, 1991). The Tennessee Value-Added Assessment System uses 
longitudinal student achievement data and national norm data (in the form of Scale Scores) to 
generate mean gain scores for teachers, schools, and districts. The mixed model methodology 
allows the model to estimate the variance of teacher effects from the average all teachers and to 
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provide a "shrinkage" estimate to assure that any attribution of effect is not based on chance. 

This model uses the gains of all students which, it is assumed, controls for variables assumed to 
contribute to achievement such as socio-economic status, family status, and school setting 
(Baker & Xu,T995; Snobgrass, 1995). 

The Dallas “value added” model was developed in the 1980s under the direction of 
Webster (Webster & Olson, 1984). A 1994 study (Webster, Mendro, & Almaguer) expanded the 
earlier study (1984). However, unlike the previous study where the predicted achievement was 
based on two years of historical standardized achievement test data, this study used many 
variables to predict individual student achievement. The basic methodology employed was to 
create predicted scores and generate standardized residuals for each outcome and predictor 
variable for each student. These two sets of residuals were then used to generate mean 
achievement above or below expected for each grade/school. The outcome variables included 
143 separate course related criterion-referenced tests, student promotion and graduation rates, 
student attendance rates, and percentage of students taking and average scores on the Scholastic 
Aptitude Test (SAT). The benefit of this methodology was the ability to investigate achievement 
independent of school identification. 

North Carolina, Louisiana, Oregon, Florida, Georgia, and Colorado are among other 
states that have invested substantial energies in reforming their accountability systems in the 
past several years. North Carolina's system (North Carolina Department of Public Instruction, 
1998) that uses an accountability approach that establishes growth expectations for each school 
(essentially a one year equivalent in growth) and then evaluates each school's progress toward 
the benchmark as evidence of accountability, may have the most to inform statistical modeling 
of any of the extant models. A similar system has been in proposed in Louisiana (Louisiana 
Department of Education, 1998). Louisiana’s system includes academic and non-academic 
indicators (attendance and dropout information) in establishing growth targets and evaluating 
progress. Criterion-baSed improvement levels are established based on required 10-year 
performance standards. 

Accountability Based on Reporting 

Many accountability systems are based on providing information to the public to make its 
own decision. These approaches usually include some type of school report cards that provide 
information about student performance. In most instances, this information is about student 
performance on standardized tests and includes comparisons to national and/or state norms. On 
of the most well-known of these is the Kentucky system. 

Previous Kentucky educational reform efforts gained national attention because of its 
systemic nature. The Kentucky reporting system is described in Assessment and Accountability: 
Report from the Prichard Committee for Academic Excellence (Task Force on Improving 
Kentucky Schools, 1995), a report that focuses on the Kentucky Instructional Results 
Information System (KIRIS). This system is primarily a reporting system to provide the results 
of the state's testing program and the progress of schools toward rewards or sanctions based on 
expected changes in student achievement. There is no "report card" other than the movement of 
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the school toward its goal. Kentucky has recently reformed its accountability system for a 
second time. The new system is called the “Commonwealth Accountability and Testing 
System” or CATS. CATS includes a reporting system and the use of norm-referenced and 
criterion-referenced assessments. The new system is intended to increase student accountability. 

A report titled School Accountability Report Cards: The Principal's Role (Stephenson, 
1989) described 13 accountability reporting procedures and school report cards. Another report 
by the Far West Regional Laboratory (Brown, 1990) described reporting systems, primarily for 
four western states. Both reports described the California School Accountability Reports. 
Accountability in Education. 

According to Brown (1990), Nevada's Report Card included educational goals and 
objectives, comparison of achievement with previous years, pupil-teacher ratio by grade, teacher 
assignments compared with qualifications and licensure, total expenditures for each source of 
funding, curriculum used including special programs, attendance and advancement records and 
graduation rates, and efforts to increase communication with parents. In the same report, 
information in Utah’s Annual School District Performance Reports included norm-referenced 
test data, ACT scores, fiscal information, attendance and dropout rates, course-taking patterns in 
high schools, professional data on teachers, and demographic figures on students. Arizona did 
not have a report card, per se, but did provide an accountability summary that discussed three 
areas-finance, program standards, and personnel. 

California School Performance Reports were differentiated by level. For the high 
schools, the reports included academic course enrollments, attendance and dropout rates, 
assessment results, and ACT/SAT scores. In the elementary and intermediate schools, the report 
was limited to achievement and attendance data. The reports were also reported separately by 
gender and race. The law also requires the state superintendent to develop a Model School 
Accountability Report Card to include 1 3 school conditions. Some of these were student 
progress toward meeting reading, writing, arithmetic, and other academic goals; progress toward 
reducing dropout rates; estimated expenditures per student and types o services funded; progress 
toward reducing class sizes and teaching loads; quality and currency of textbooks and other 
instructional material; availability of qualified personnel to provide student support services; 
adequacy of school facilities; adequacy of teacher evaluations and professional improvement 
opportunities; classroom discipline and climate for learning; curriculum improvement 
programs; and quality of school instruction and leadership. 

The standards, indicators of quality, data collection procedures, and instruments for the 
evaluation of Georgia public schools were discussed in a paper presented in 1991 (Tesh, 1991, 
April). The paper recommended that data elements be considered over time. Once the Georgia 
Comprehensive Evaluation System (CES) is fully implemented, each school was to be visited 
and reviewed every five years. This review considered the status of the many indicators across 
the years since the last site visit rather than limiting consideration to the outcomes of a single 
year. The outcome goals of education were improving student's school completion rate; 
preparing students for post-secondary school life (preparing students for continued education, 
preparing students for work, and preparing students for multiple life roles); and providing 
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students with school experiences appropriate to their ages, developmental levels and skill levels 
(educating elementary school students, educating middle school students, educating secondary 
school students, and increasing special education students' living skills and opportunities). 

To address these goals the following indicators were to be collected and reported: 
Improving student's school completion rate (including drop out rate, graduation rate, initial pass 
rates on the Georgia basic skills test, minority initial pass rate on the Georgia basic skills test, 
and percent minority scoring in the lowest quartile on the ITBS/TAP); preparing students for 
post-secondary school life (including preparing students for continued education, the world of 
work, and multiple life roles); and providing students with school experiences appropriate to 
their ages, developmental levels and skill levels. 

In addition, the system looks at the characteristics of effective schools and reports the 
following indicators: The autonomy of school-site management, the school principal’s level of 
instructional leadership to the school, the stability of the school staff, parental involvement and 
support of the school, school promotion and school-wide recognition of academic achievement 
of students, organization of the school environment with respect to the maximization of learning 
time, the collegiality of relationships between and among teachers and administrators as it 
relates to collaborative planning, student and staff sense of belonging to a supportive 
community, school culture as it presents clear goals and high expectations for students, and the 
orderliness and safety of the school environment. 

Since 1996, Georgia has released state, school and district report cards which include 
data on norm- and criterion-referenced assessments. Scholastic Assessment Test performance, 
data on attendance, drop-out, teacher characteristics, school characteristics, funding, and 
community characteristics. The expressed purposes of the reporting system are school 
improvement and informing the public about educational quality at local and state levels. 

School improvement grants are available. Georgia is in the process of developing a 
computerized criterion-referenced assessment system to measure student achievement on the 
proposed Quality Core Curriculum. Teachers will be able to use such tests periodically to 
students for diagnostic/remedial purposes. 

San Diego Schools evaluated 10 “Leadership in Accountability Schools” in that district 
(Bennett, 1995, April). This model included four components of accountability; (1) standards, 
(2) related assessments, (3) recognition and intervention measures, and (4) public reporting 
practices. The issue of concern within this project was the public reporting practices. "Schools 
will fully and accurately report student achievement results in a public process that emphasizes 
progress achieved." Schools referenced a wide variety of evidence when reporting on their 
performance in reporting results, but validation reviewers indicated that most of the information 
was shared orally. Some schools used materials such as parent bulletins, newsletters, brochures, 
minutes from meetings, student handbooks, charts and lists of committees and governance 
groups. Top schools have well-defined, two-way systems of communication among stakeholders 
about student achievement and progress toward the school goals and expectations. The 
reporting is based on site-developed accountability systems. Therefore, this report does not 




5 



include elements for reporting, particularly elements that may be directly or indirectly related to 
the achievement levels reported to the public. 

Summary 

It is clear that, what are called accountability models, range from just reporting results to 
comparing actual achievement to predicted achievement. The models vary in their complexity 
and their purpose. The one common element in all of the accountability models is a reporting 
system. State-based accountability models also tend to apply sanctions for “academic” 
bankruptcy, financial incentives for reaching or exceeding expectations, and funds for 
improvement at school or district levels. This background was the basis for the development of 
the new school accountability model by the Center for Educational Accountability at the 
University of Alabama at Birmingham. 

Existing accountability systems have primarily served functions of reporting to 
stakeholders, monitoring school and district performance on a common set of indicators across 
time, identifying and sanctioning schools and districts that are failing to meet expectations, and 
providing financial incentives for improvement or exemplary performance. These models have 
not however been diagnostic-remedial in their focus. That is, they have been moderately 
effective in identifying strong versus weak educational performance, but they have not focused 
on identifying those conditions, characteristics, or processes that are associated with differential 
performance between schools. Therefore, the stakeholders are left with an awareness of the 
schools' performance status or change but not data-based directions for improving school 
performance. This is a significant limitation. It is the difference between as status-focused 
accountability model and an improvement-focused model. We believe that there is a need for a 
paradigmatic shift to improvement-focused accountability modeling. 

In order to generate improvement-focused accountability models there must be explicit 
recognition of the context within which school performance and school improvement occurs. 

We have adopted a simple systems perspective for our preliminary phase of model development. 
Such simple systems reflect the dynamic nature of schools and districts while allowing for 
simple longitudinal modeling of input-process-output cycles. By identifying the inputs and 
processes that are reliably related to differences in outputs AND can be manipulated we hope to 
provide recommendations for improving educational systems and documenting intended and 
unintended consequences of such interventions. 

Describing the School Accountability Model 

With the general accountability movement in mind, personnel at the Center for 
Educational Accountability at the University of Alabama at Birmingham designed a model to 
address the shortcomings of other models. First, a set of characteristics were developed. These 
are a school accountability model should: 
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1 . provide for the external judgment of a school and its programs based on known 
standards (preferably, world class), 

2. provide information to school personnel and parents for diagnostic/remedial 
efforts focused on improvement, 

3. provide an opportunity for the school to communicate its goals and achievements 
to parents and the public, 

4. be culturally fair in the sense that the model does not unduly penalize racial or 
gender groups, and 

5. be systemic in nature. 

Implementing such a system would allow schools to be held accountable for their performance 
while using the information to continually improve the learning of their students. Often, new 
programs are judged based on how the participants, instructors, or developers feel about them. 
Hard evidence of their effectiveness is seldom available and when it is, it often of poor quality. 
A systemic model would provide for the prediction of changes throughout the system when a 
new element is introduced. 

The first step in development of the model was identifying the educational inputs, 
processes, and outcomes of the school. Ideally, this would be done by the school’s teachers, 
administrators, and parents. For the purposes of our research, this was done by committees of 
teachers and school administrators, using the information from the Alabama School Report 
Cards as a starting point. Table 1 illustrates some of the elements in the model. Two 
ingredients that will impact the success of the model are the way in which the data are analyzed 
and the method by which the results are reported. 

At this time, we anticipate that the data will be analyzed using structural equation 
modeling. Structural equation modeling is a statistical process that tests the fit of the data to a 
preconceived model and determines the odds ratios (probabilities) for moving from one element 
to another. Figure 1 illustrates the theoretical form of the model with only two variables 
associated with each component. In Figure 1, rectangular figures represent observable variables 
and oval figures represent latent variables, or theoretical variables that account for the 
information in one or more observable variables. Once a model is fitted and tested, it will 
permit school personnel to explore the impact of changes before they are implemented. 

Information will be reported using a School Accountability Profile. The Profile will 
permit the school to share a broad picture of its accomplishments with parents and the 
community. The School Accountability Profile will also provide a basis of judging past 
accomplishments and planning future improvements. If it were produced annually, the impact 
of changes in the school could be judged based on empirical evidence. 



Table 1 



Example Inputs, Processes, and Outcomes in the Ec 


ucational Process 


INPUTS 


PROCESSES 


OUTCOMES 


✓Financial (e.g., 
expenditures per average 
daily attendance; local, 
state, and federal revenue 
per average daily 
attendance) 

✓Personnel (e.g., pupil 
teacher ratio, average 
teacher salary) 
✓Facilities 
✓Equipment 
✓Materials 
✓School Policy/Law 
✓Student Attributes (e.g., 
predominant socio- 
economic level, average 
ability) 


✓Curriculum & Instruction 
✓Implementation of Policies 
(e.g., admission, 
grading, promotion, 
etc.) 

✓Diverse Educational 
Opportunities 
✓Parent Involvement 
✓Leadership (planning, 
style, efficiency, etc.) 


✓Academic achievement 
(e.g., SAT, exit exam, 
writing test) 

✓Accomplishments (e.g., 
graduate rates, college 
attendance) 

✓Attitudes 

✓Retention/dropout rates 
✓School safety 
✓Discipline 

✓College enrollment and 
completion rates 
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Figure 1: Illustrative School Accountability Model 
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The School Accountability Model is currently undergoing pilot testing in two of the three 
largest school systems in the state: Birmingham City Schools and Jefferson County Schools. 
Between these two school systems, there are schools representing almost every demographic in 
the state: from urban to rural, economically depressed to wealthy, and small to large. The 
database that forms the core of a data warehouse to run the system is developed, and data are 
being input from multiple sources. The bulk of the data are being obtained electronically from 
the Alabama State Department of Education. The reduces the time teachers and school 
administrators must spend gathering data and the time Center personnel must spend entering 
data. Other data are being obtained from the school systems’ central office. Thus, very little 
data must be collected directly from the schools. 

The School Accountability Profile, eventually, will include five years of data (see 
Appendix A). Since the current standardized tests used in Alabama (the Stanford Achievement 
Tests or SAT, 9 th Edition) were first used in 1996, the first Profiles will have only data for three 
years. 
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SCHOOL ACCOUNTABILITY PROFILE 

I 1 

: ! SCHOOL: Allan Cott School 

SYSTEM: Jefferson 

i j 

| ! For School Year 1995-96 

; i 
I ; 

! ! Schedule 



School Goals 



i ■ ■ 

. i 

i 

| ! Major Accomplishments 




BEST COPY AVAILABLE 

Produced by the 

Center for Educational Accountability 
University of Alabama at Birmingham 
901 1 3 th Street South 
Birmingham, AL 35294-1250 
205/934-7598 Fax 205/975-5389 
http://WWW.UAB.EDU/educ/CEA.HTM 
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School: Allan Cott School 



Variable/Year 95^96 9^97 97^98 9&99 9M0 

^iool 

School ID 

County or System 

County or System ID 

School Age 

Economics 

Reduced Lunches 

Free Lunches 

Per Capita Expenditures 

Portables 

Enrollment 

Total 

Ex-Ed 

Parent Teacher Association 

Parents 

Faculty 

Parent Teacher Conference 
Fall 

Students Represented 
Number of Parents 
Spring 

Students Represented 
Number of Parents 
Open House Attendance 
Library Circulation 
Faculty and Staff 

Phone Calls . • 

Absences 
Job Related 
Non-Job Related 
Staff Development 
Volunteerism 
Number 
Hours 
Awards 

Certificated Teachers 

Classified Staff 

Students 

Gender 

Males 

Females 

Ethnicity 

African-American 

Asia 11 

Caucasian BEST COPY AVAILABLE 

Hispanic 

American Indian or Alaskan Native 
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School: Allan Cott School 



Variable/Year 95-96 96-97 97-98 98-99 99-00 

tnrollment by Grade 

k 

1 

2 

3 

4 

5 

Attendance by Grade 

k 

1 

2 

3 

4 

5 

Promotion Rate 
Close of School 
Summer School 
Retention Rate 
Close of School 
Summer School 
Ex-Ed 

Enrollment by Grade . 

k 

1 % 

2 I ' •. • 

3 v 

4 < v 

5 ’’ ■ ... • 

Attendance by Grade 

k 

1 

2 

3 

4 

5 

Promotion Rate 
Close of School 
Summer School 
Retention Rate 
Close of School 
Summer School 
Dropout Rate 

Discipline Events (occurances) 

Class 1 
Class 2 
Class 3 

In-school Suspension 
Out-of-school Suspension 
Assigned to Alternative School 
Out-of-school Expulsion 

Tardies \ £ 

Activities 

0 
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School: 


Allan Cott School 


Variable/Year 95.96 


96-97 97-98 98-99 99-00 


Honor Koil 

First Grading Period 

Second Grading Period 

Third Grading Period 

Fourth Grading Period 

Grade 5 Writing 

Narrative 

Level 1 

Level 2 

Level 3 

Level 4 

Descriptive 

Level 1 

Level 2 

Level 3 

Level 4 

Expository 

Level 1 

Level 2 

Level 3 

Level 4 

Stanford Achievement Test by Grade 

Grade 3 

Reading 

Mathematics 

Language 

Science 

Social Science 

Battery 

Grade 4 

Reading 

Mathematics 

Language 

Science 

Social Science 

Battery 

Grades 

Reading 

Mathematics 

Language 

Science 

Social Science 

Battery 

Survey Results 

Strengths 

Certificated Staff 

Classified Staff 

Parents 

Students 

Concerns 


BEST COPY AVAILABLE 


Certificated Staff 
Classified Staff 
Parents 
0 Students 
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